Permutational Rademacher Complexity - A New Complexity Measure for Transductive Learning
نویسندگان
چکیده
Abstract. Transductive learning considers situations when a learner observes m labelled training points and u unlabelled test points with the final goal of giving correct answers for the test points. This paper introduces a new complexity measure for transductive learning called Permutational Rademacher Complexity (PRC) and studies its properties. A novel symmetrization inequality is proved, which shows that PRC provides a tighter control over expected suprema of empirical processes compared to what happens in the standard i.i.d. setting. A number of comparison results are also provided, which show the relation between PRC and other popular complexity measures used in statistical learning theory, including Rademacher complexity and Transductive Rademacher Complexity (TRC). We argue that PRC is a more suitable complexity measure for transductive learning. Finally, these results are combined with a standard concentration argument to provide novel data-dependent risk bounds for transductive learning.
منابع مشابه
Transductive Rademacher Complexity and Its Applications
We develop a technique for deriving data-dependent error bounds for transductive learning algorithms based on transductive Rademacher complexity. Our technique is based on a novel general error bound for transduction in terms of transductive Rademacher complexity, together with a novel bounding technique for Rademacher averages for particular algorithms, in terms of their “unlabeled-labeled” re...
متن کاملTransductive Rademacher Complexities for Learning Over a Graph
Recent investigations [12, 2, 8, 5, 6] and [11, 9] indicate the use of a probabilistic (’learning’) perspective of tasks defined on a single graph, as opposed to the traditional algorithmical (’computational’) point of view. This note discusses the use of Rademacher complexities in this setting, and illustrates the use of Kruskal’s algorithm for transductive inference based on a nearest neighbo...
متن کاملHuman Rademacher Complexity
We propose to use Rademacher complexity, originally developed in computational learning theory, as a measure of human learning capacity. Rademacher complexity measures a learner’s ability to fit random labels, and can be used to bound the learner’s true error based on the observed training sample error. We first review the definition of Rademacher complexity and its generalization bound. We the...
متن کاملBounds for Learning the Kernel: Rademacher Chaos Complexity
In this paper we develop a novel probabilistic generalization bound for regularized kernel learning algorithms. First, we show that generalization analysis of kernel learning algorithms reduces to investigation of the suprema of homogeneous Rademacher chaos process of order two over candidate kernels, which we refer to it as Rademacher chaos complexity. Our new methodology is based on the princ...
متن کاملRelax and Localize: From Value to Algorithms
We show a principled way of deriving online learning algorithms from a minimax analysis. Various upper bounds on the minimax value, previously thought to be non-constructive, are shown to yield algorithms. This allows us to seamlessly recover known methods and to derive new ones. Our framework also captures such “unorthodox” methods as Follow the Perturbed Leader and the R forecaster. We emphas...
متن کامل